Multi-modal machine learning medical assessment

ABSTRACT

Apparatuses, systems, methods, and computer program products are disclosed for multi-modal machine learning medical assessment. A source module is configured to receive multiple types of data for a user. A machine learning module is configured to analyze the multiple types of data using machine learning to determine multiple predictions of likelihoods of the user getting a neurological disease. A multi-modal result module configured to determine a single result indicating a likelihood of the user getting the neurological disease based on the multiple predictions.

FIELD

This invention relates to medical assessments and more particularlyrelates to a multi-modal machine learning medical assessment.

BACKGROUND

Early detection of neurological illnesses such as Alzheimer's can bedifficult. However, when detected early, progress of a neurologicalillness can often be treated and/or slowed.

SUMMARY

Apparatuses are presented for multi-modal machine learning medicalassessment. In one embodiment, a source module is configured to receivemultiple types of data for a user. A machine learning module, in certainembodiments, is configured to analyze the multiple types of data usingmachine learning to determine multiple predictions of likelihoods of theuser getting a neurological disease. In a further embodiment, amulti-modal result module is configured to determine a single resultindicating a likelihood of the user getting the neurological diseasebased on the multiple predictions.

An apparatus, in another embodiment, includes means for receivingmultiple types of data for a user. In some embodiments, an apparatusincludes means for analyzing the multiple types of data using machinelearning to determine multiple predictions of likelihoods of the usergetting a neurological disease. An apparatus, in certain embodiments,includes means for determining a single result indicating a likelihoodof the user getting the neurological disease based on the multiplepredictions.

Methods are presented for multi-modal machine learning medicalassessment. In one embodiment, a method includes receiving multipletypes of data for a user. In a further embodiment, a method includesanalyzing the multiple types of data using machine learning to determinemultiple predictions of likelihoods of the user getting a neurologicaldisease. A method, in some embodiments, includes determining a singleresult indicating a likelihood of the user getting the neurologicaldisease based on the multiple predictions.

Computer program products comprising a computer readable storage mediumare presented. In certain embodiments, a computer readable storagemedium stores computer usable program code executable to performoperations for multi-modal machine learning medical assessment. In someembodiments, one or more of the operations may be substantially similarto one or more steps described above with regard to the disclosedapparatuses, systems, and/or methods.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the invention will be readilyunderstood, a more particular description of the invention brieflydescribed above will be rendered by reference to specific embodimentsthat are illustrated in the appended drawings. Understanding that thesedrawings depict only typical embodiments of the invention and are nottherefore to be considered to be limiting of its scope, the inventionwill be described and explained with additional specificity and detailthrough the use of the accompanying drawings, in which:

FIG. 1 is a schematic block diagram illustrating one embodiment of asystem for multi-modal machine learning medical assessment;

FIG. 2 is a schematic block diagram illustrating one embodiment of anassessment module;

FIG. 3 is a schematic block diagram illustrating a further embodiment ofa system for multi-modal machine learning medical assessment;

FIG. 4 is a schematic flowchart diagram illustrating one embodiment of amethod for multi-modal machine learning medical assessment; and

FIG. 5 is a schematic flowchart diagram illustrating a furtherembodiment of a method for multi-modal machine learning medicalassessment.

DETAILED DESCRIPTION

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, appearances of the phrases“in one embodiment,” “in an embodiment,” and similar language throughoutthis specification may, but do not necessarily, all refer to the sameembodiment, but mean “one or more but not all embodiments” unlessexpressly specified otherwise. The terms “including,” “comprising,”“having,” and variations thereof mean “including but not limited to”unless expressly specified otherwise. An enumerated listing of itemsdoes not imply that any or all of the items are mutually exclusiveand/or mutually inclusive, unless expressly specified otherwise. Theterms “a,” “an,” and “the” also refer to “one or more” unless expresslyspecified otherwise.

Furthermore, the described features, advantages, and characteristics ofthe embodiments may be combined in any suitable manner. One skilled inthe relevant art will recognize that the embodiments may be practicedwithout one or more of the specific features or advantages of aparticular embodiment. In other instances, additional features andadvantages may be recognized in certain embodiments that may not bepresent in all embodiments.

These features and advantages of the embodiments will become more fullyapparent from the following description and appended claims, or may belearned by the practice of embodiments as set forth hereinafter. As willbe appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method, and/or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module,” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having program code embodied thereon.

Many of the functional units described in this specification have beenlabeled as modules (or components), in order to more particularlyemphasize their implementation independence. For example, a module maybe implemented as a hardware circuit comprising custom VLSI circuits orgate arrays, off-the-shelf semiconductors such as logic chips,transistors, or other discrete components. A module may also beimplemented in programmable hardware devices such as field programmablegate arrays, programmable array logic, programmable logic devices or thelike.

Modules may also be implemented in software for execution by varioustypes of processors. An identified module of program code may, forinstance, comprise one or more physical or logical blocks of computerinstructions which may, for instance, be organized as an object,procedure, or function. Nevertheless, the executables of an identifiedmodule need not be physically located together, but may comprisedisparate instructions stored in different locations which, when joinedlogically together, comprise the module and achieve the stated purposefor the module.

Indeed, a module of program code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data may be identified and illustrated hereinwithin modules, and may be embodied in any suitable form and organizedwithin any suitable type of data structure. The operational data may becollected as a single data set, or may be distributed over differentlocations including over different storage devices, and may exist, atleast partially, merely as electronic signals on a system or network.Where a module or portions of a module are implemented in software, theprogram code may be stored and/or propagated on in one or more computerreadable medium(s).

The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (“RAM”), aread-only memory (“ROM”), an erasable programmable read-only memory(“EPROM” or Flash memory), a static random access memory (“SRAM”), aportable compact disc read-only memory (“CD-ROM”), a digital versatiledisk (“DVD”), a memory stick, a floppy disk, a mechanically encodeddevice such as punch-cards or raised structures in a groove havinginstructions recorded thereon, and any suitable combination of theforegoing. A computer readable storage medium, as used herein, is not tobe construed as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The schematic flowchart diagrams and/or schematic block diagrams in theFigures illustrate the architecture, functionality, and operation ofpossible embodiments of apparatuses, systems, methods and computerprogram products according to various embodiments of the presentinvention. In this regard, each block in the schematic flowchartdiagrams and/or schematic block diagrams may represent a module,segment, or portion of code, which comprises one or more executableinstructions of the program code for implementing the specified logicalfunction(s).

It should also be noted that, in some alternative embodiments, thefunctions noted in the block may occur out of the order noted in theFigures. For example, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved. Other steps and methods may be conceived that are equivalentin function, logic, or effect to one or more blocks, or portionsthereof, of the illustrated Figures.

Although various arrow types and line types may be employed in theflowchart and/or block diagrams, they are understood not to limit thescope of the corresponding embodiments. Indeed, some arrows or otherconnectors may be used to indicate only the logical flow of the depictedembodiment. For instance, an arrow may indicate a waiting or monitoringperiod of unspecified duration between enumerated steps of the depictedembodiment. It will also be noted that each block of the block diagramsand/or flowchart diagrams, and combinations of blocks in the blockdiagrams and/or flowchart diagrams, can be implemented by specialpurpose hardware-based systems that perform the specified functions oracts, or combinations of special purpose hardware and program code.

FIG. 1 depicts one embodiment of a system 100 for multi-modal machinelearning medical assessment. In one embodiment, the system 100 includesone or more hardware computing devices 102, one or more assessmentmodules 104 (e.g., one or more assessment modules 104 a disposed on theone or more hardware computing devices 102, one or more backendassessment modules 104 b, or the like), one or more data networks 106 orother communication channels, and/or one or more backend server devices108. In certain embodiments, even though a specific number of hardwarecomputing devices 102, assessment modules 104, data networks 106, and/orbackend server devices 108 are depicted in FIG. 1 , one of skill in theart will recognize, in light of this disclosure, that any number ofhardware computing devices 102, assessment modules 104, data networks106, and/or backend server devices 108 may be included in the system 100for multi-modal machine learning medical assessment.

In general, an assessment module 104, in various embodiments, isconfigured to receive and/or collect multiple types of data (e.g., imagedata, volumetric data, evaluation data, or the like) for a user (e.g., apatient, another user, or the like) and/or to assess and/or diagnose thepresence and/or severity of one or more neurological diseases (e.g.,Alzheimer's disease, Parkinson's disease, Multiple Sclerosis,Amyotrophic Lateral Sclerosis (ALS), or the like) based on machinelearning analyses for the multiple types of data. An assessment module104 may receive data from a user, from a medical professional evaluatinga user, from a computing device 102, from a backend server device 108,over a data network 106, from one or more user interface elements of ahardware computing device 102 and/or a backend server device 108, or thelike. An assessment module 104 may provide a single, combined result(e.g., an assessment, or the like) to a user (e.g., a patient, a medicalprofessional, or the like).

Based on a machine learning analysis of the multiple types of data, anassessment module 104 may assess and/or diagnose a likelihood of a usergetting one or more neurological diseases or other medical conditions.For example, after an assessment module 104 receives multiple types ofdata for a user, an assessment module 104 may analyze and/or score thedifferent types of data using machine learning to determine multiplepredictions of likelihoods of the user getting a neurological disease,determine a single result indicating a likelihood of the user getting aneurological disease based on the multiple predictions, and/or mayprovide the single result to a user (e.g., the user being evaluated, amedical professional, or the like) in a user interface of an electronicdisplay screen of a hardware computing device 102, through anapplication programming interface (API), or the like. An assessmentmodule 104 may analyze the multiple types of data using one or moremachine learning models trained for a certain neurological diseaseand/or other medical condition.

An assessment module 104 may compare a user's data to previous data fromwhen the user was healthy (e.g., to baseline data). An assessment module104 may normalize results based on a user's demographic (e.g., age,gender, or the like). An assessment module 104 may determinenormalization data as part of training process, determining a range ofexpected scores for each demographic, or the like.

In certain embodiments, an assessment module 104 determining whether ornot a user is likely to get a neurological disease may facilitate earlydetection of the neurological disease, allowing more effective treatmentof the neurological disease, better quality of life for the user, or thelike. An assessment module 104 using multiple modalities, or types ofdata, in some embodiments, may provide a more accurate assessment and/orother result than a single modality of data alone, may have fewer falsenegatives and/or false positives than a single modality of data alone,or the like.

For example, an assessment module 104 may analyze image data such asmagnetic resonance imaging (Mill) images, X-ray images, or the like of abrain of the user, volumetric data (e.g., volumetric measurements of abrain of the user), evaluation data for the user from a medicalprofessional (e.g., a clinical diagnosis or other evaluation accountingfor age, genetic indicators, or the like), and/or other modalities ortypes of data using machine learning to determine multiple predictionsof likelihoods of the user getting a neurological disease, such asAlzheimer's disease or the like. An assessment module 104 may determinea single result based on the multiple predictions (e.g., a single and/orfinal clinical dementia rating (CDR), or the like).

An assessment module 104 may receive multiple types of data for a userand process the multiple types of data to determine if a person has amedical condition, such as a neurological disease. For example, anassessment module 104 may process the multiple types of data to computea yes or no determination as to whether the person has the medicalcondition or to compute a score that indicates a probability or alikelihood that the person has the medical condition and/or a severityof the medical condition.

As used herein, a diagnosis or other result relates to any determinationas to whether a person may have a medical condition or any determinationas to a possible severity of the medical condition. A diagnosis or otherresult may include any form of an assessment, conclusion, opinion, ordetermination relating to a medical condition. In some instances, adiagnosis or other result may be incorrect, and a person diagnosed witha medical condition may not actually have the medical condition.

An assessment module 104 may receive the data for a user using anyappropriate techniques. For example, a frontend assessment module 104 amay be installed as an application or “app” on a hardware computingdevice 102 that uses a REST (representational state transfer) API callto transmit the multiple types of data over the internet, a mobiletelephone network, or other data network 106 to a backend assessmentmodule 104 b installed on a backend server device 108. In anotherexample, a medical professional may have a hardware computing device 102that is used to record data for a person and transmit the data to anassessment module 104. In some embodiments, an assessment module 104 maybe installed on a hardware computing device 102 such that it is notnecessary to transmit the data over a data network 106.

An assessment module 104 may process multiple types of data with machinelearning to perform a medical diagnosis. In processing the multipletypes of data, features may be computed from the multiple types of data,and then the features may be processed by the machine learning. Anyappropriate type of features may be used.

To train a machine learning model for diagnosing a medical condition, acorpus of training data may be collected. The training corpus mayinclude examples of data where the diagnosis of the person is known. Forexample, it may be known that the person had no Alzheimer's disease, ora mild, moderate, or severe case of Alzheimer's disease. An assessmentmodule 104 may use a training corpus that includes multiple types ofdata for training a machine learning model for diagnosing Alzheimer'sdisease and/or another neurological disease.

In one embodiment, the system 100 includes one or more hardwarecomputing devices 102. The hardware computing devices 102 and/or the oneor more backend server devices 108 (e.g., computing devices, informationhandling devices, or the like) may include one or more of a desktopcomputer, a laptop computer, a mobile device, a tablet computer, a smartphone, a set-top box, a gaming console, a smart TV, a smart watch, afitness band, an optical head-mounted display (e.g., a virtual realityheadset, smart glasses, or the like), an HDMI or other electronicdisplay dongle, a personal digital assistant, and/or another computingdevice comprising a processor (e.g., a central processing unit (CPU), aprocessor core, a field programmable gate array (FPGA) or otherprogrammable logic, an application specific integrated circuit (ASIC), acontroller, a microcontroller, and/or another semiconductor integratedcircuit device), a volatile memory, and/or a non-volatile storagemedium. In certain embodiments, the hardware computing devices 102 arein communication with one or more backend server devices 108 via a datanetwork 106, described below. The hardware computing devices 102, in afurther embodiment, are capable of executing various programs, programcode, applications, instructions, functions, or the like.

In various embodiments, an assessment module 104 may be embodied ashardware, software, or some combination of hardware and software. In oneembodiment, an assessment module 104 may comprise executable programcode stored on a non-transitory computer readable storage medium forexecution on a processor of a hardware computing device 102; a backendserver device 108; or the like. For example, an assessment module 104may be embodied as executable program code executing on one or more of ahardware computing device 102; a backend server device 108; acombination of one or more of the foregoing; or the like. In such anembodiment, the various modules that perform the operations of anassessment module 104, as described below, may be located on a hardwarecomputing device 102; a backend server device 108; a combination of thetwo; and/or the like.

In various embodiments, an assessment module 104 may be embodied as ahardware appliance that can be installed or deployed on a backend serverdevice 108, on a user's hardware computing device 102 (e.g., a dongle, aprotective case for a phone 102 or tablet 102 that includes one or moresemiconductor integrated circuit devices within the case incommunication with the phone 102 or tablet 102 wirelessly and/or over adata port such as USB or a proprietary communications port, or anotherperipheral device), or elsewhere on the data network 106 and/orcollocated with a user's hardware computing device 102. In certainembodiments, an assessment module 104 may comprise a hardware devicesuch as a secure hardware dongle or other hardware appliance device(e.g., a set-top box, a network appliance, or the like) that attaches toanother hardware computing device 102, such as a laptop computer, aserver, a tablet computer, a smart phone, or the like, either by a wiredconnection (e.g., a USB connection) or a wireless connection (e.g.,Bluetooth®, Wi-Fi®, near-field communication (NFC), or the like); thatattaches to an electronic display device (e.g., a television or monitorusing an HDMI port, a DisplayPort port, a Mini DisplayPort port, VGAport, DVI port, or the like); that operates substantially independentlyon a data network 106; or the like. A hardware appliance of anassessment module 104 may comprise a power interface, a wired and/orwireless network interface, a graphical interface (e.g., a graphics cardand/or GPU with one or more display ports) that outputs to a displaydevice, and/or a semiconductor integrated circuit device as describedbelow, configured to perform the functions described herein with regardto an assessment module 104.

An assessment module 104, in such an embodiment, may comprise asemiconductor integrated circuit device (e.g., one or more chips, die,or other discrete logic hardware), or the like, such as afield-programmable gate array (FPGA) or other programmable logic,firmware for an FPGA or other programmable logic, microcode forexecution on a microcontroller, an application-specific integratedcircuit (ASIC), a processor, a processor core, or the like. In oneembodiment, an assessment module 104 may be mounted on a printed circuitboard with one or more electrical lines or connections (e.g., tovolatile memory, a non-volatile storage medium, a network interface, aperipheral device, a graphical/display interface. The hardware appliancemay include one or more pins, pads, or other electrical connectionsconfigured to send and receive data (e.g., in communication with one ormore electrical lines of a printed circuit board or the like), and oneor more hardware circuits and/or other electrical circuits configured toperform various functions of an assessment module 104.

The semiconductor integrated circuit device or other hardware applianceof an assessment module 104, in certain embodiments, comprises and/or iscommunicatively coupled to one or more volatile memory media, which mayinclude but is not limited to: random access memory (RAM), dynamic RAM(DRAM), cache, or the like. In one embodiment, the semiconductorintegrated circuit device or other hardware appliance of an assessmentmodule 104 comprises and/or is communicatively coupled to one or morenon-volatile memory media, which may include but is not limited to: NANDflash memory, NOR flash memory, nano random access memory (nano RAM orNRAM), nanocrystal wire-based memory, silicon-oxide based sub-10nanometer process memory, graphene memory,Silicon-Oxide-Nitride-Oxide-Silicon (SONOS), resistive RAM (RRAM),programmable metallization cell (PMC), conductive-bridging RAM (CBRAM),magneto-resistive RAM (MRAM), dynamic RAM (DRAM), phase change RAM (PRAMor PCM), magnetic storage media (e.g., hard disk, tape), optical storagemedia, or the like.

The data network 106, in one embodiment, includes a digitalcommunication network that transmits digital communications. The datanetwork 106 may include a wireless network, such as a wireless cellularnetwork, a local wireless network, such as a Wi-Fi network, a Bluetooth®network, a near-field communication (NFC) network, an ad hoc network,and/or the like. The data network 106 may include a wide area network(WAN), a storage area network (SAN), a local area network (LAN), anoptical fiber network, the internet, or other digital communicationnetwork. The data network 106 may include two or more networks. The datanetwork 106 may include one or more servers, routers, switches, and/orother networking equipment. The data network 106 may also include one ormore computer readable storage media, such as a hard disk drive, anoptical drive, non-volatile memory, RAM, or the like.

The one or more backend server devices 108, in one embodiment, mayinclude one or more network accessible computing systems such as one ormore web servers hosting one or more web sites, an enterprise intranetsystem, an application server, an API server, an authentication server,or the like. A backend server device 108 may include one or more serverslocated remotely from the hardware computing devices 102. A backendserver device 108 may include at least a portion of the assessmentmodules 104, may comprise hardware of an assessment module 104, maystore executable program code of an assessment module 104 in one or morenon-transitory computer readable storage media, and/or may otherwiseperform one or more of the various operations of an assessment module104 described herein for shared content tracking and attribution.

FIG. 2 depicts one embodiment of an assessment module 104. Theassessment module 104, in certain embodiments, may be substantiallysimilar to one or more of a device assessment module 104 a and/or abackend assessment module 104 b, as described above with regard to FIG.1 . The assessment module 104, in the depicted embodiment, includes asource module 202, a machine learning module 204, a multi-modal resultmodule 206, and an interface module 208.

In one embodiment, the source module 202 is configured to receivemultiple types of data for a user. For example, in various embodiments,a source module 202 may receive image data of a brain of a user (e.g.,MRI images, X-ray images, computerized tomography (CT) scan images, orthe like), volumetric data (e.g., volumetric measurements, estimates, orthe like) for a brain of a user, evaluation data for a user (e.g.,clinical evaluation data from a doctor, nurse, and/or other medicalprofessional, or the like), and/or other types of data relevant to aneurological disease or other medical condition.

In certain embodiments, a plurality of source modules 202 disposed on aplurality of different computing devices 102 may receive data for aplurality of different users. For example, a plurality of distributedsource modules 202 may collect data samples for a medical trial, totrain a machine learning model for diagnosing a medical condition, orthe like.

The source module 202, in one embodiment, may store received multipletypes of data on a computer readable storage medium of a computingdevice 102, 110, so that the machine learning module 204 may accessand/or process the received multiple types of data to diagnose and/orassess a medical condition, train a machine learning model fordiagnosing and/or assessing a medical condition, or the like; so thatthe interface module 208 may provide the received multiple types of datato one or more authorized users; and/or so that the received multipletypes of data are otherwise accessible for use. In another embodiment,the source module 202 may provide received multiple types of datadirectly to the machine learning module 204 for diagnosing and/orassessing a medical condition (e.g., without otherwise storing the data,temporarily storing and/or caching the data, or the like). The sourcemodule 202 may store and/or organize received types of data in adatabase and/or other predefined data structure accessible by themachine learning module 204, the multi-modal result module 206, theinterface module 208, or the like.

By storing multiple types of data for a user, in certain embodiments,the source module 202 may enable the machine learning module 204 and/orthe multi-modal result module 206 to dynamically assess a medicalcondition for the user. For example, the source module 202 may storemultiple types of data for a user on a hardware computing device 102, ona backend server device 108 in communication with a hardware computingdevice 102 over a data network 106, or the like, enabling the machinelearning module 204 to determine an assessment of a medical.

In one embodiment, a machine learning module 204 is configured toanalyze multiple types of data from a source module 202 using machinelearning to determine multiple predictions of likelihoods of a usergetting a neurological disease and/or other medical condition. Forexample, in some embodiments, a machine learning module 204 may usedifferent machine learning models to process different types of data(e.g., a convolutional neural network comprising a residual neuralnetwork or the like to analyze image data of a brain of a user, a Knearest neighbor classifier to analyze volumetric data of a brain of auser, a random forest to analyze evaluation data of a user, or thelike).

In one embodiment, the machine learning module 204 may determinemultiple assessments or other predictions of a neurological disease orother medical condition for a user (e.g., indicating whether or not theuser has the medical condition, a likelihood that the user will have themedical condition, an estimated severity of the medical condition, orthe like), each based on a different type of data received for the user.The machine learning module 204, in certain embodiments, may determinebinary predictions of whether a user is likely to get a neurologicaldisease or other medical condition (e.g., yes or no, likely or unlikely,positive or negative, or the like).

In one embodiment a multi-modal result module 206 is configured todetermine a single assessment and/or other result indicating alikelihood of a user getting a neurological disease or other medicalcondition based on multiple types of data from a source module 202and/or multiple predictions from a machine learning module 204, or thelike. For example, the multi-modal result module 206 may combine orotherwise process and/or analyze predictions for multiple types of datafor a user, into a single assessment or other result indicating whetheror not the user is likely to get a neurological disease or other medicalcondition.

The multi-modal result module 206 may use one or more rules to determinea single assessment and/or other result based on multiple predictions,multiple types of data, or the like. In one embodiment, the multi-modalresult module 206 may use a conservative rule, and may be configured todetermine that a user is likely to get a neurological disease or othermedical condition in response to all of multiple predictions indicatingthe user is likely to get the neurological disease or other medicalcondition.

In a further embodiment, the multi-modal result module 206 may use avoting rule, and may be configured to determine that a user is likely toget a neurological disease or other medical condition in response to amajority of multiple predictions indicating the user is likely to getthe neurological disease or other medical condition (e.g., at least twoout of three, three out of four or five, four out of six or seven, fiveout of eight or nine, six out of ten or eleven, or the like).

In one embodiment, the multi-modal result module 206 may use anaggressive rule, and may be configured to determine that a user islikely to get a neurological disease or other medical condition inresponse to at least one of multiple predictions indicating that theuser is likely to get the neurological disease or other medicalcondition (e.g., if any one type of data indicated that the user islikely to get the neurological disease, the multi-modal result module206 determines that the user is likely to get the neurological disease).

In certain embodiments, the multi-modal result module 206 may use anoverride rule allowing a prediction based on one type of data tooverride one or more other predictions, types of data, or the like. Forexample, the multi-modal result module 206 may be configured todetermine that a user is likely to get a neurological disease or othermedical condition if a predefined type of data indicates that it islikely, if two predefined types of data indicate that it is likely, orthe like. In one embodiment, the multi-modal result module 206 may beconfigured to determine that a user is likely to get a neurologicaldisease or other medical condition in response to either an evaluationof the user by a medical professional indicating the user is likely toget the neurological disease or other medical condition (e.g., as anoverride type of data), or both image data of a brain of the user andvolumetric data for the brain of the user indicating the user is likelyto get the neurological disease or other medical condition (e.g.,allowing two other types of data to counter the override type of data).

In some embodiments, instead of or in addition to one or more of theabove rules, the multi-modal result module 206 may be configured to usea machine learning analysis to determine a single assessment or otherresult indicating a likelihood of the user getting a neurologicaldisease or other medical condition (e.g., by processing the multipletypes of data for the user from the source module 202 and/or thepredictions from the machine learning module 204 as inputs into amachine learning model, such as a decision tree or other machinelearning model, which may provide the single assessment or otherresult).

The interface module 208, in certain embodiments, is configured toexecute on a hardware computing device 102 (e.g., of a user such as amedical professional evaluating a patient, a patient, of the like)and/or on a backend server device 108, or the like. In one embodiment,the interface module 208 may be configured to provide a user interfaceto a medical professional, a patient, and/or to another user. In afurther embodiment, the interface module 208 is configured to provide anAPI to the source module 202, the machine learning module 204, themulti-modal result module 206, other interface modules 208, otherassessment modules 104, hardware computing devices 102, backend serverdevices 108, or the like.

The interface module 208, in one embodiment, is configured to cooperatewith the source module 202, the machine learning module 204, and/or themulti-modal result module 206. For example, the source module 202 may beconfigured to receive multiple types of data for a user through a userinterface of the interface module 208 displayed on an electronic displayscreen of a hardware computing device 102 and to provide the multipletypes of data to the machine learning module 204 using an API of theinterface module 208, or the like. In a further example, the interfacemodule 208 may be configured to receive a single assessment or otherresult indicating a likelihood of a user getting a neurological diseaseand/or another medical condition from the multi-modal result module 206over an API of the interface module 208, and the interface module 208may be configured to display the single result in a user interface on anelectronic display screen of a hardware computing device 102.

In one embodiment, the interface module 208 provides one or more userswith access to received types of data from the source module 202 (e.g.,image data, volumetric data, evaluation data, or the like), topredictions and/or other results from the machine learning module 204and/or the multi-modal result module 206, or the like. The interfacemodule 208 may allow a user to access received types of data,predictions and/or other results, or the like from multiple locations(e.g., from a mobile app on a mobile computing device 102, from a webbrowser of a different computing device 102 accessing a web server of abackend server device 108, or the like).

In certain embodiments, the interface module 208 may enforce accesscontrol permissions (e.g., for privacy, for security, for HIPAAcompliance, or the like) by authenticating users (e.g., with a usernameand password or other authentication credentials) and providing theusers access to types of data, predictions or other results, or the likebased on access control permissions associated with the user.

FIG. 3 depicts one embodiment of a system 300 for multi-modal machinelearning medical assessment. In the depicted embodiment, the sourcemodule 202 receives image data 302, volumetric data 304, and evaluationdata 306 and provides the received types of data 302, 304, 306 to themachine learning module 204. The machine learning module 204 analyzesthe image data 302 using a neural network 308 (e.g., a convolutionalneural network comprising a residual neural network, or the like) todetermine a first prediction, analyzes the volumetric data 304 using a Knearest neighbor classifier 310 to determine a second prediction, andanalyzes the evaluation data 306 using a random forest 312 to determinea third prediction.

In the depicted embodiment, the multi-modal result module 206 analyzesand/or combines the first, second, and third predictions of likelihoodsof a user getting a neurological disease or other medical condition fromthe neural network 308, the K nearest neighbor classifier 310, and therandom forest 312 to determine a single result 314 indicating alikelihood of the user getting the neurological disease or other medicalcondition (e.g., using a conservative rule, a voting rule, an aggressiverule, an override rule, a decision tree, or the like). The multi-modalresult module 206 provides the result 314 to the interface module 208.The interface module 208 may display the result 314 to a user (e.g., amedical professional, a patient, and/or another user) on an electronicdisplay screen of a hardware computing device 102, may provide theresult 314 over an API in response to an API request, or the like.

FIG. 4 depicts one embodiment of a method 400 for multi-modal machinelearning medical assessment. The method 400 begins, and a source module202 receives 402 multiple types of data for a user. A machine learningmodule 204 analyzes 404 the multiple types of data using machinelearning to determine multiple predictions of likelihoods of the usergetting a neurological disease or other medical condition. A multi-modalresult module 206 determines 406 a single result indicating a likelihoodof the user getting the neurological disease or other medical conditionbased on the multiple predictions and the method 400 ends.

FIG. 5 depicts one embodiment of a method 500 for multi-modal machinelearning medical assessment. The method 500 begins, and a source module202 receives 502 image data 302 of a brain of a user. A machine learningmodule 204 analyzes 504 the image data 302 using a neural network 308and determines 506 a prediction of the likelihood of the user getting aneurological disease.

The source module 202 receives 508 volumetric data 304 for the brain ofthe user. The machine learning module 204 analyzes 510 the volumetricdata 304 using a K nearest neighbor classifier 310 and determines 512 aprediction of the likelihood of the user getting the neurologicaldisease. The source module 202 receives 514 an evaluation 306 of theuser by a medical professional. The machine learning module 204 analyzes516 the evaluation data 306 using a random forest 312 and determines 518a prediction of the likelihood of the user getting the neurologicaldisease.

A multi-modal result module 206 combines 520 the predictions 506, 512,518 into a single result 314 indicating a likelihood of the user gettingthe neurological disease (e.g., using a conservative rule, a votingrule, an aggressive rule, an override rule, a decision tree, or thelike). An interface module 208 displays 522 the single result 314 in auser interface on an electronic display screen of a hardware computingdevice 102 and the method 500 ends.

A means for receiving multiple types of data for a user, in variousembodiments, may comprise an assessment module 104, a source module 202,an interface module 208, a hardware computing device 102, a hardwareserver device 108, a data network 106, an MM device, an X-ray device, aCT scanner device, a camera or other optical sensor, a microphone orother audio sensor, another sensor device, a user interface, an API, akeyboard device, a network interface, a mobile application, a processor,an application specific integrated circuit (ASIC), a field programmablegate array (FPGA), programmable logic, other logic hardware, and/orother executable program code stored on a non-transitory computerreadable storage medium. Other embodiments may comprise substantiallysimilar or equivalent means for receiving multiple types of data for auser.

A means for analyzing multiple types of data using machine learning todetermine multiple predictions of likelihoods of a user getting aneurological disease, in various embodiments, may comprise an assessmentmodule 104, a machine learning module 204, a multi-modal result module206, a hardware computing device 102, a hardware server device 108, aneural network 308, a K nearest neighbor classifier 310, a random forest312, a mobile application, a processor, an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA),programmable logic, other logic hardware, and/or other executableprogram code stored on a non-transitory computer readable storagemedium. Other embodiments may comprise substantially similar orequivalent means for analyzing multiple types of data using machinelearning to determine multiple predictions of likelihoods of a usergetting a neurological disease.

A means for determining a single result indicating a likelihood of auser getting a neurological disease based on multiple predictions, invarious embodiments, may comprise an assessment module 104, a machinelearning module 204, a multi-modal result module 206, a hardwarecomputing device 102, a hardware server device 108, a decision tree, amobile application, a processor, an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA), programmablelogic, other logic hardware, and/or other executable program code storedon a non-transitory computer readable storage medium. Other embodimentsmay comprise substantially similar or equivalent means for determining asingle result indicating a likelihood of a user getting a neurologicaldisease based on multiple predictions.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. An apparatus comprising: a source moduleconfigured to receive multiple types of data for a user; a machinelearning module configured to analyze the multiple types of data usingmachine learning to determine multiple predictions of likelihoods of theuser getting a neurological disease; and a multi-modal result moduleconfigured to determine a single result indicating a likelihood of theuser getting the neurological disease based on the multiple predictions.2. The apparatus of claim 1, wherein at least one of the multiple typesof data comprises image data of a brain of the user.
 3. The apparatus ofclaim 2, wherein the image data comprises magnetic resonance images ofthe brain of the user.
 4. The apparatus of claim 2, wherein at least oneof the multiple types of data comprises volumetric data for the brain ofthe user.
 5. The apparatus of claim 4, wherein at least one of themultiple types of data comprises an evaluation of the user by a medicalprofessional.
 6. The apparatus of claim 5, wherein the machine learningmodule is configured to analyze the evaluation of the user by a medicalprofessional using a random forest, to analyze the volumetric data ofthe brain of the user using K nearest neighbors, and to analyze theimage data of the brain of the user using a convolutional neural networkcomprising a residual neural network.
 7. The apparatus of claim 1,wherein the multi-modal result module is configured to determine theuser is likely to get the neurological disease in response to one ormore of the evaluation of the user by the medical professionalindicating the user is likely to get the neurological disease, and boththe image data of the brain of the user and the volumetric data for thebrain of the user indicating the user is likely to get the neurologicaldisease.
 8. The apparatus of claim 1, wherein the multi-modal resultmodule is configured to determine the user is likely to get theneurological disease in response to at least one of the multiplepredictions indicating the user is likely to get the neurologicaldisease.
 9. The apparatus of claim 1, wherein the multi-modal resultmodule is configured to determine the user is likely to get theneurological disease in response to a majority of the multiplepredictions indicating the user is likely to get the neurologicaldisease.
 10. The apparatus of claim 1, wherein the multi-modal resultmodule is configured to determine the user is likely to get theneurological disease in response to all of the multiple predictionsindicating the user is likely to get the neurological disease.
 11. Theapparatus of claim 1, wherein the multi-modal result module isconfigured to determine the single result indicating the likelihood ofthe user getting the neurological disease by processing the multipletypes of data for the user with machine learning comprising a decisiontree.
 12. The apparatus of claim 1, wherein the neurological diseasecomprises Alzheimer's disease.
 13. The apparatus of claim 1, furthercomprising an interface module configured to execute on a computingdevice of a medical professional evaluating the user.
 14. The apparatusof claim 13, wherein the source module is configured to receive themultiple types of data for the user through a user interface of theinterface module displayed on an electronic display screen of thecomputing device and to provide the multiple types of data to themachine learning module using an application programming interface. 15.The apparatus of claim 14, wherein the interface module is configured toreceive the single result indicating the likelihood of the user gettingthe neurological disease from the multi-modal result module over theapplication programming interface and to display the single result inthe user interface on the electronic display screen of the computingdevice.
 16. An apparatus comprising: means for receiving multiple typesof data for a user; means for analyzing the multiple types of data usingmachine learning to determine multiple predictions of likelihoods of theuser getting a neurological disease; and means for determining a singleresult indicating a likelihood of the user getting the neurologicaldisease based on the multiple predictions.
 17. The apparatus of claim16, wherein the multiple types of data for the user comprise image dataof a brain of the user, volumetric data for the brain of the user, andan evaluation of the user by a medical professional, and wherein themachine learning comprises a random forest to analyze the evaluation ofthe user by the medical professional, a K nearest neighbor classifier toanalyze the volumetric data of the brain of the user, and aconvolutional neural network comprising a residual neural network toanalyze the image data of the brain.
 18. A method comprising: receivingmultiple types of data for a user; analyzing the multiple types of datausing machine learning to determine multiple predictions of likelihoodsof the user getting a neurological disease; and determining a singleresult indicating a likelihood of the user getting the neurologicaldisease based on the multiple predictions.
 19. The method of claim 18,wherein the multiple types of data for the user comprise image data of abrain of the user, volumetric data for the brain of the user, and anevaluation of the user by a medical professional.
 20. The method ofclaim 19, wherein the machine learning comprises a random forest toanalyze the evaluation of the user by the medical professional, a Knearest neighbor classifier to analyze the volumetric data of the brainof the user, and a convolutional neural network comprising a residualneural network to analyze the image data of the brain.